Evaluating Middle Years Students’ Proportional Reasoning 
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Proportional reasoning is a key aspect of numeracy that is not always developed naturally by 
students. Understanding the types of proportional reasoning that students apply to different 
problem types is a useful first step to identifying ways to support teachers and students to 
develop proportional reasoning in the classroom. This paper describes the development of a 
diagnostic instrument that aims to identify situations in which students can apply 
proportional reasoning and the types of reasoning they use. 

Proportional reasoning is mathematical reasoning involving a sense of co-variation and 
multiple comparisons (Lesh, Post, & Behr, 1988). It requires multiplicative and relational 
thinking, which do not always develop naturally in students (Sowder et al., 1998). Lesh et 
al. (1988) described proportional reasoning as the cornerstone of higher level areas of 
mathematics, such as algebra, and the capstone of elementary concepts, such as arithmetic, 
number, and measurement. The importance of proportional reasoning goes beyond the 
mathematics classroom; its importance in other areas of study, for example, science, 
economics, and demographics has been widely recognized (Akatugba & Wallace, 2009; 
Boyer, Levine, & Huttenlocher, 2008). Perhaps more importantly, proportional reasoning is 
an essential aspect of everyday applications of numeracy. According to Ahl, Moore, and 
Dixon (1992), proportional reasoning is a “pervasive activity that transcends topical barriers 
in adult life” (p. 81) and yet, it has been estimated that more than half of the adult 
population are not proportional thinkers (Lamon, 1999). 

Until recently, little was known about students’ development or application of 
proportional reasoning (Lamon, 2005). Establishing the types and accuracy of reasoning 
used by students in situations of ratio and proportion can assist teachers in selecting 
teaching strategies and learning activities to target students’ learning needs and strengthen 
their ability to reason proportionally (Misailidou & Williams, 2003). This paper describes 
the development of a diagnostic instrument designed to identity the types of reasoning used 
by students in a range of proportional and non-proportional situations and presents results 
from the pilot instrument to illustrate its use in diagnosing students’ strategies and 
understanding. 


Rationale 

Several researchers have highlighted the importance of establishing children’s 
proportional reasoning skills (e.g., Bright, Joyner, & Wallis, 2003; Misailidou & Williams, 
2003; Van Dooren, De Bock, Hessels, Janssens, & Verschaffel, 2005). Methods that can be 
used to determine students’ understanding include interviews, pen-and-paper tests (open- 
ended or multiple choice items), and concept mapping (Tuysiiz, 2009). According to 
Wiggins and McTighe (1998), assessment must require students to explain or defend their 
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answer. While interviews provide a powerful method of establishing students’ 
understanding, this is impractical in many situations, for example, in research situations 
where large numbers of students are involved. Tamir (1989) found that asking students for 
justifications when answering multiple choice questions is a sensitive and effective means 
of assessing students’ understanding and argued that such an approach addresses some 
limitations commonly associated with traditional multiple-choice questions. To account for 
these arguments, Treagust (1995) recommended the use of two-tiered multiple-choice 
instruments as an alternative to individual interviews as a means of obtaining information 
about the reasoning of groups of students. The first tier of a two-tiered item consists of a 
multiple-choice question, with two to four choices. The second tier requires students to 
choose from four reasons to justify or explain their answer to the first tier question. To be 
considered correct, students must answer both levels of the question correctly. The data 
generated provide an insight into the common reasoning strategies employed by classes or 
year level cohorts. 

For a number of years, science education researchers have designed and successfully 
used two-tiered multiple-choice instruments to diagnose students’ understanding in a 
number of topics, as a means of informing pedagogical strategies (Chandrasegaran, 
Treagust, & Mocerino, 2008; Haslam & Treagust, 1987; Ozmen, 2008; Tan & Treagust, 
1999; Treagust, 1995, 2006; Tiiysuz, 2009). O’Keefe and O’Donoghue (2011) used a two- 
tiered instrument based on Treagust’s work to investigate the effectiveness of an 
intervention strategy in lower secondary mathematics classrooms; however, they did not 
provide examples of the items used. To date, such instruments have not been widely used 
for investigating students’ understanding in mathematics. 

Method 


Instrument Design 

The design of many two-tiered items is based on findings reported in research literature 
about students’ common alternative conceptions, or the errors and difficulties they 
commonly encounter in a particular topic, concept, or reasoning situation. In the case of the 
instrument described here, research literature and findings from a previous study by 
members of the research team informed both the choice of problem type and the nature of 
the reasoning responses in the second tier of each item. 

A detailed review of the literature in this area revealed varied descriptions of 
proportional reasoning problem types and the circumstances in which students typically use 
reasoning strategies correctly and incorrectly. A brief overview is provided here. Lamon 
(1993) identified four types of proportion problems: well-chunked or well-known measures 
(e.g., relationships that are commonly used rates, such as speed); part-part-whole (e.g., ratio 
problems in which two complementary parts are compared with each other or the whole); 
associated sets (rate situations in which the relationship between quantities is defined within 
the question); and stretchers and shrinkers (growth or scale problems). Lamon’s categories 
formed the basis of the item types, however, Lesh et al. (1988) described several problem 
types that are neglected in textbooks, instruction, and research (e.g., problems involving 
between or within representation translations and missing value problems). Further, Van 
Dooren et al. (2005) suggested that often, students rely on proportional reasoning in 
circumstances that do not require it (e.g., constant, linear, and additive situations). 
Following a similar line of reasoning, Bright et al. (2003) stated that in order to assess 
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students’ ability to reason proportionally, it is important to provide situations in which 
students can correctly and incorrectly use multiplicative and additive thinking. 

The items on the instrument were chosen to account for these arguments and 
suggestions and were of the following types: 

• Non-proportional (constant or additive situations) 

• Part-part-whole, missing value problems 

• Scale (one- and two-dimensional) 

• Rate (well-chunked measures, between representation translation) 

• Relative - absolute (associated sets, within and between representation translation) 

• Inverse proportion (well-chunked measures, associated sets) 

Nine of the twelve items were based on items from the Keeping it in Proportion test 
(KIIP), used in a study by Dole, Clarke, Wright, and Hilton (2007). In that study, over 800 
students (Years 5-9) completed the KIIP test, which employed short response questions that 
required students to show their working and provide written explanations of their calculated 
answers. These problems were re-worded to present a scenario and make a statement in the 
first tier, to which students responded True ox False. To ensure that the instrument described 
here included items that targeted constant and additive non-proportional situations and two- 
dimensional scale, three additional items were based on problem types used by Bright et al. 
(2003) and Van Dooren et al. (2005). 

The responses in the second tier of each item were based on the common strategies used 
by students in situations of proportion described in the literature. For example, 
multiplicative thinking is the critical factor for the comparison of quantities in proportion- 
related tasks. In contrast, additive reasoning involves considering the sums or differences in 
quantities (Bright et al., 2003). Students often find it difficult to discern which reasoning to 
use in a particular situation. Other difficulties associated with proportional situations and the 
errors commonly made by students that have been identified by researchers include 

• an inability to discern when to use proportional reasoning and difficulty in 
identifying multiplicative or relative relationships (Van De Walle, Karp, & Bay- 
Williams, 2010); 

• a tendency to approach proportional situations additively instead of multiplicatively 
(Cramer & Post, 1993; Lamon, 1993; Misailidou & Williams, 2003; Tourniaire & 
Pulos, 1985); 

• the use of multiplicative approaches unnecessarily, for example, when additive 
strategies should be used (Van Dooren, De Bock, & Verschaffel, 2010); 

• inappropriate use of algorithms, such as cross multiplication (Nabors, 2003); and 

• incorrect build-up/pattem building (Lamon, 1993; Misailidou & Williams, 2003). 

In the instrument described here, second tier responses were used to identify students’ 
correct and incorrect application of additive and multiplicative thinking, their recognition of 
absolute and relative situations, their ability to distinguish situations of proportion from non- 
proportion, and the strategies they used. In addition to the literature around students’ 
difficulties in situations of proportion, the findings of the study by Dole et al. (2007) in 
which the KIIP test were used informed the wording of some options in the second tier of 
the items. 
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To illustrate the two-tier items, an example is provided. In the item, students were asked 
to compare two washing powders and to respond to the statement “Powder A is the better 
value”. The students were provided with a graphic that portrayed the following information: 

Powder A comes in 1 kg containers that cost $4 for 20 loads of washing. 

Powder B comes in 1.5 kg containers that cost $6.50 for 30 loads of washing. 

The second tier responses were 

1 . Washing powder A costs the least. 

2. Washing powder B costs a little bit more but you get 10 more loads of washing. 

3. The cost per load of washing is less. 

4. Both washing powders are the same value. 

This problem requires students to used relative thinking in a situation involving an 
associated set. Option 1 indicates that the student has used absolute thinking, comparing 
only price. Option 2 shows that the students have considered only the number of loads and 
not the price. Option 3 is the most accurate response and shows that students have used 
relative thinking. Option 4 suggests that students are considering the mass and number of 
loads but not the cost in their reasoning. 

Administration of the Pilot Instrument 

Prior to administering the instrument to the pilot group, the items were reviewed and 
revised by members of the research team (who have extensive mathematics teaching 
experience in primary and/or secondary schools) and a sample of teachers. The pilot 
instrument was administered to 140 Year 5 and 6 students (11-12 years old) in composite 
classes in two primary schools. Three classes of students from each school completed the 
instrument. Prior to administration of the instrument, the school principals were briefed on 
the purpose and administration procedure for the instrument. The teachers were provided 
with written instruction regarding the its administration. 

The students’ responses were coded and the percentage of students who responded to 
each alternative was calculated. The results of each class, school and the whole group were 
compared to determine whether the results were consistent or whether there were anomalies 
in the data. The results were similar for each group. 

The combinations of responses to the first and second tiers allow identification of the 
students’ reasoning in each item. Tan and Treagust (1999) suggested that when interpreting 
such data for the purpose of gaining an understanding of students’ misconceptions, it is 
reasonable to consider those response combinations that exist for at least 10 per cent of 
students. The incorrect responses for which the percentage of students was greater than 10 
per cent were further investigated to determine the types of reasoning used by the students. 

Results 

The data for the full sample of 140 students, shown in Table 1, show that the majority of 
students were able to answer Item 1 and just over half answered Item 7 correctly, however, 
for a number of items, a high percentage of students chose an incorrect combination. 
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Table 1 

Percentage of Year 5 and 6 Students (n = 140) Selecting Each Response Combination 


[ tem Response 



TA 

FA 

TB 

FB 

TC 

FC 

TD 

FD 

1 

13.6 

2.1 

2.1 

0.7 

3.6 

70.0* 

3.6 

0.7 

2 

5.0 

5.0 

8.6 

45.7* 

21.4 

5.0 

5.7 

2.9 

3 

3.6 

47.9 

12.1 

9.3 

6.4 

13.6 

5.0* 

2.9 

4 

1.4 

9.3 

3.6 

32.9 

14.3* 

10.7 

2.9 

23.6 

5 

12.1 

5.7 

0.7 

34.3* 

1.4 

6.4 

2.1 

33.6 

6 

15.7 

1.4 

4.3 

42.9 

27.9* 

2.1 

0.7 

2.1 

7 

5.0 

17.1 

0.0 

52.1* 

6.4 

8.6 

5.0 

6.4 

8 

2.9 

23.6* 

2.9 

10.0 

2.9 

14.3 

37.9 

10.7 

9 

58.6 

9.3 

3.6 

10.7 

4.3 

0.0 

0.7 

12.9* 

10 

4.3 

2.1* 

78.6 

2.1 

7.1 

0.7 

5.0 

0.0 

11 

33.6 

13.6 

20.0* 

7.1 

4.3 

10.0 

0.7 

7.1 

12 

0.7 

21.4 

4.3 

13.6 

19.3* 

4.3 

5.7 

29.3 


Notes. *indicates the correct response combination for the item. TA, FA, etc. denote first and second tier 
response combinations. 


Examination of the combinations of incorrect responses selected by at least 10 per cent 
of the students allowed identification of the reasoning used in each situation. In some 
circumstances, the reasoning used indicated that the students had some level of qualitative 
understanding of the situation or relationship. The strategies used are shown in Table 2. 


Table 2 

Common Incorrect Strategies Used by Students 


Strategy or error 

Choice 

% 

Some qualitative understanding of relationship 

Understands need to compare part with whole 

Item 3 (FC) 

13.9 

Qualitative understanding of speed-time relationship 

Item 5 (FD) 

33.6 

Qualitative understanding of inverse relationship 

Item 8 (D) 

48.6 

Some evidence of qualitative understanding of linear scale 

Item 9 (FB) 

10.7 

Qualitative understanding of part-part-whole 

Item 7 (FA) 

17.1 

Making absolute comparison instead of relative comparison 
Considering one value without considering the total 

Item 3 (TB, FA) 

60.0 

Considering one value without considering a related value 

Item 6 (TA) 

15.7 


Item 12 (FA, FB) 

35.0 

Reading scale as absolute, not considering value of units 

Item 9 (TA) 

58.6 
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Using multiplicative strategies inappropriately 

Applying a multiplicative strategy to a constant situation 

Item 1 (TA) 

13.6 

Applying a multiplicative strategy to an additive situation 

Item 2 (TC) 

21.4 

Treating an inverse situation as if it involved direct proportion 

Item 5 (TA) 

12.1 

Incorrect reasoning but recognizes inverse situation 

Item 8 (FC) 

14.3 

No recognition of impact of changing two dimensions on area 

Item 10 (TB) 

78.6 

Using additive strategies in proportional situation 

Increases two quantities by the same amount (direct) 

Item 4 (FB, FD) 

56.5 

Increases two quantities by the same amount (inverse) 

Item 8 (FB) 

10.0 

Erroneous calculation / no calculation 

Inaccurately uses multiplicative reasoning 

Item 4 (FC) 

10.7 

Not calculating a value/estimating 

Item 6 (FB) 

42.9 

Faulty understanding of visual representations 

Interprets graph accurately but chooses irrelevant response 

Item 1 1 (A) 

47.2 

Misinterprets a distance-time graph 

Item 1 1 (FC) 

10.0 

Misinterprets relative amounts in pictorial representation 

Item 12 (FD) 

29.3 


Discussion and Conclusion 

The results of the pilot instrument showed that it was useful for identifying the students’ 
reasoning and the areas in which they applied incorrect reasoning strategies. The findings 
also aligned with the findings in the research regarding common errors and problems 
encountered by students in situations of proportion. As indicated by the data in Table 1, the 
students’ success on the items varied. While many students answered some items correctly, 
the majority of students found particular items challenging, for example, Item 3 (requiring 
relative thinking) and Item 10 (requiring understanding of two-dimensional enlargement). 

A major problem for students appears to be discriminating non-proportional from 
proportional situations. For example, for both non-proportional situations (Items 1 and 2), 
the majority of students who chose incorrectly used multiplicative thinking. Similarly, in 
proportional situations (Items 4 and 8), the majority of incorrect responses indicated the use 
of additive approaches. In addition to inappropriately using additive and multiplicative 
strategies, many students used absolute comparison in situations requiring relative thinking. 
For example, in several items, they only considered one variable when it was necessary to 
compare two parts or one part to the whole (Items 3, 6, 9). 

It is important to note that in several items (Items 3, 5, 7, 8, 9), a number of students 
responded in ways that indicated a level of qualitative understanding of the relationship 
involved. This perhaps suggests that more targeted work with such students might assist 
them to develop a deeper understanding. 

The data from the pilot instrument are for students in Years 5 and 6. As the final 
instrument is intended for use across Years 5 to 9, the items were designed to allow 
discrimination across year levels. This was found to be the case and may account for the low 
scores on some of the pilot items. For example, situations of inverse proportion (Items 5 & 
8) are not usually a focus for students in primary years. Based on the data and feedback 
from the pilot, the original 12 items were retained for the final instrument. The items were 
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arranged randomly so that no consecutive items were of the same problem type. Although 
some minor rewording was applied to make the meaning of the question clearer, the items 
and the second tier responses were retained. 

The proportional reasoning two-tiered instrument described in this paper is a new 
approach to assessing students’ proportional reasoning. The data generated from two-tiered 
instruments allows students’ common errors or partial understanding to be identified, which 
informs researchers and teachers about the areas that teachers might target to enhance 
students’ proportional reasoning. For example, those items that indicate students’ use of 
absolute comparisons when they should make relative comparisons infonn teachers that this 
is an area of weakness in their students. Such data may also be used to inform the design of 
teacher professional development to promote teachers’ understanding of the different types 
of proportional reasoning and classroom strategies to address their students’ needs. 

The instrument described in this paper may be used as a pre-test and post-test to allow 
the researchers and teachers to track students’ progress and to identify changes in their 
proportional reasoning to inform teachers’ future curriculum planning and practice. In light 
of the importance of proportional reasoning development and the difficulties students 
encounter when reasoning proportionally, the various applications of this instrument have 
the potential to benefit teachers, curriculum planners, researchers and teacher educators. 
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